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ABSTRACT 



A plan was developed to improve test scores in a Georgia 
high school. A committee of teachers, students, citizens, and a curriculum 
consultant was formed to analyze the school 1 s problems and the reason for low 
test scores and low achievement. Data on the school climate indicated that it 
was poor, except that the instructional leadership of the administration and 
the quality of instruction as rated by teachers were considered good. The 
educational climate was considered deficient in openness to new ideas, 
values, and beliefs, and in involvement in the decision-making process. The 
school district's central office had already decided that the type of 
instruction had to change, so teachers were asked to choose among four 
alternative teaching strategies: integrated instruction, cooperative 
learning, learning styles, and the Paideia approach. They settled on 
implementing integrated instruction, cooperative learning, and the 
recognition of learning styles. Seminars and activities were conducted on 
team-building and shared decision making, and inservice training in these 
methodologies was provided. Pretest data from 1994-1995 and posttest data 
from the end of the 1995-96 school year measured progress related to scores 
on the Tests of Achievement and Proficiency (TAP) for approximately 150 
students. Test scores did improve, as did scores for school climate on a 
measure developed for the study. Discipline problems appeared to be less 
frequent, although there was no improvement in student attendance for 
absences over 10 days. Test scores and teacher feedback indicate that the 
school improvement program was effective. Appendixes contain three figures of 
school data and a survey. (Contains four tables, three figures, and five 
references.) (SLD) 
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DETERMINING THE EFFECTIVENESS OF A SCHOOL IMPROVEMENT PLAN 



Introduction 

Suppose you are a high school principal and the superintendent tells you that you have 
to do something about test scores at your school. He tells you that the problem is the 
teachers who bore students because all they do is lecture. He wants that to stop and gives 
you a mandate to embark on a school improvement project that will improve test scores. 

What can you do? Top down change is difficult at best! As you reflect on how to proceed, 
you remember a strategy you learned in your graduate class for dealing with curriculum 
problems (Eisner, 1994; Longstreet & Shane, 1993). The strategy consisted of four steps: 1) 
analyze the problem; 2) design a plan to solve to the problem; 3) implement the plan; and 4) 
evaluate the results of the plan. In order to conduct the analysis, a committee was formed to 
decide how to attack the problem. 

A committee of teachers, students, citizens, and a curriculum consultant from a 
neighboring university was assembled to analyze the problem. The committee decided that 
they would write a grant ($5,000.00) to get funds for the analysis and design stages. 

Funding for the grant was received from the Georgia State Department of Education. As 
part of the analysis stage, the committee decided that test scores and school climate data on 
the high school were necessary. After reviewing different climate instruments, the Tennessee 
School Climate Inventory (TSCI) and the Group Openness and Trust Scale (GOTS) were 
selected. These climate instruments had been used in another school improvement project 
with good results (Benton and Bulach, 1995). The committee, based on the research of 
Bulach, Malone, & Castleman, 1995, believed that school climate could be important for 
improving student achievement. 

The climate data revealed that the overall climate of the school was very poor with 
two exceptions: 1) the instructional leadership of the administration and 2) the quality of 
instruction provided by the teachers were viewed positively by the staff. The climate was 
very poor in two areas. They were: 1) openness to new ideas, values, and beliefs and 2) 
involvement in the decision-making process. 

The test data that was collected verified that scores were indeed below the state and 
national averages. Exceptions were ninth and tenth grade science scores and tenth grade 
social studies scores. The overall averages, however, were below state and national 
averages. 

After careful analysis of the data, the committee established the improvement of 
school climate and test scores as goals for the high school. The curriculum consultant 
identified two other concerns for the committee to consider. They were the following: 1) the 
high climate scores on instructional leadership and the quality of staff instruction indicated 
that their teaching methodologies were perceived as effective; and 2) their low climate scores 
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on openness and lack of involvement in the decision-making process could result in 
adversarial relationships. Consequently, it was believed that there would be a great deal of 
staff resistance on attempts to change the instructional process. Their high scores on 
instruction and low scores on openness suggested that they felt instruction was already good, 
and they would not be receptive to new ideas. Members of the committee confirmed this and 
added that the teachers were angry at being told that they had to change the way they were 
teaching. Further, teachers believed that they would have little to say about the instructional 
improvement process. Based on these beliefs, the committee realized that any plan would 
have to involve the teachers in the process as much as possible. 

The problem was that since the central office had already decided that the type of 
instruction had to change, how could teachers believe they had some say in the process? The 
committee believed that the design of the plan would have to give teachers some feeling of 
control if it were to be successful. Consequently, it was decided that each department would 
be allowed to select a new teaching strategy of their preference to implement. In other 
words, while they had to change, they at least had some control over the type of change. 
Further they did not have to change their teaching completely. The teaching methodology 
they chose could be used to supplement their existing teaching style. 

Selecting the new teaching methodology was the next step. Teachers were surveyed 
and asked to rank their preference from a list of teaching strategies. Four teaching strategies 
were identified: integrated instruction, cooperative learning, learning styles, and the Padeia 
approach. 

The next step was to have each department identify which of these four new teaching 
strategies they wanted to learn. It was believed that collegial support was necessary. This 
would be more likely to occur if everyone in a department learned the same teaching new 
strategy. Visitations were made by selected staff members to other schools where these 
teaching methodologies were being used. Further, one staff development day was used to 
bring in consultants who gave an overview of each of the four teaching methodologies. As a 
result of these experiences, the Paideia approach was dropped since it required everyone in 
the school to agree on the approach. The Paideia approach would not allow each department 
to do what they wanted. The three other teaching methodologies were to be implemented the 
following school year. 

Purpose of the School Improvement Plan (SIP1 

Based on the above, the committee decided to write another grant to secure funds 
($10,000.00) to implement a plan that would: 1) train staff in each department in these 
teaching methodologies; and 2) improve school climate. Based on success in achieving these 
two goals, members of the School Improvement Committee believed that test scores and 
climate scores would improve. Three factors that could have affected student achievement 
and climate scores were absenteeism, office discipline referrals, and suspensions. It was 
believed that the school improvement plan would positively affect these factors. An 
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improvement in these areas should positively affect climate and student test scores. 
Consequently, collecting data on these variables was part of the school improvement plan as 
well. The grant was approved, but due to a delay in funding from the Georgia State 
Department of Education, the school improvement plan was not implemented until February, 



1996. 



The plan to improve school climate consisted of seminars and activities on team- 
building and shared decision-making. The plan to improve test scores included five days of 
in-service on the teaching methodologies chosen by the various departments, e.g., 
cooperative learning, learning styles, and integrated thematic teaching. 

The series of seminars and other activities focused on team-building and shared 
decision-making, involved ten hours of training in human relations, ten hours of training in 
conflict management, and ten hours of training in group dynamics. The seminars were 
attended by representatives selected from each department. A one-day team-building activity 
was held at the end of the 95-96 school year and was attended by the entire staff. 
Additionally, a school climate committee was formed to plan activities for the staff designed 
to improve school climate. Faculty barbecues, breakfasts prepared by administrators, 
decorating of halls, pep rallies, making the lounge more appealing, and getting help with the 
copying were some of the planned activities. 

Activities designed to improve school-based decision-making were integrated 
throughout the plan. For example, there were: the school improvement planning committee; 
each department selecting a teaching strategy; the seminars; and the team building activity. 
These activities were also designed to improve school based decision-making and school 
climate. 

Due to the delay in funding, most of the training took place over the summer of 1996. 
Consequently, the complete effect of the school improvement plan (SIP) was not felt until the 
96-97 school year. The data to measure the effectiveness of the plan were collected at the 
end of the 95-96 school year. This data will be collected again at the end of the 96-97 
school year for a more valid measure of the effectiveness of this school improvement plan. 

Results 

Pre- and post-data were collected on the following indicators to measure progress 
related to test scores: Tests of Achievement and Proficiency (TAP) scores, student 
absenteeism, suspensions, and office referrals. A t-test for independent groups was used on 
TAP scores to determine if the differences between the pre- and post-school improvement 
plan data were statistically significant. 

TAP scores were collected on students who were in grade 9 and grade 10 during the 
94-95 school year (pre-data). These scores were compared against their performance while 
in grade 10 and grade 11 during the 95-96 school year (post-data). The pre- and post-data 
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scores were compared for gains or losses during the 95-96 school year, i.e., the year of the 
school improvement grant. Appendices A and B are graphs that illustrate the results of these 
comparisons. Except for changes in enrollment due to withdrawals or new enrollees, these 
are the same students. The grade 10 and grade 11 students (95-96), however, were exposed 
to teachers involved in a school improvement plan while the grade 9 and grade 10 students 
(94-95) were not. The TAP scores show an improvement in all categories except science 
scores (see Appendix A). The tenth grade students (94-95) when compared with their 
performance as eleventh grade students (95-96) show a similar pattern. There is an 
improvement in all areas but one-reading (see Appendix B). The average score for all 
subject areas is higher than the average for the prior year. It would appear from this data that 
the plan to improve the quality of instruction and student test scores was successful. 

A further check on TAP scores was calculated to control for withdrawals and new 
enrollees. The only grades for which test data were available for all students were grades 9 
and 10. Consequently, these were the only grades measured by this procedure. Students in 
grade 9 during the 94-95 school year who were also enrolled during the 95-96 school year 
were compared on their composite TAP test scores using the t-test for independent groups. 
Students who withdrew before the 95-96 school year or who enrolled during the 95-96 school 
year were excluded from this comparison. The mean composite score on the TAP for these 
students as ninth graders was 53.6 compared to 54.4 as tenth graders (see table # 1). The t- 
score to test the significance of the difference for this comparison was .74 (p > .05). This 
showed that while there was a positive difference in scores it was not significant. The unit 
of analysis for this comparison was individual students. The unit of analysis for all other 
comparisons in this study was the school. 

Table # 1. A comparison of TAP scores before and during the SIP. 



Students 


Mean 

Score 


Standard 

Deviation 


t-score 


Probability 


Grade 9 pre- 


53.6 


28.0 


.73 


.46 


Grade 10 post- 


54.4 


28.9 







n = 150 
df = 148 

The data on student absenteeism were collected on all grades for the entire school for 
the 94-95 and 95-96 school year. The enrollment increased from 890 during the 94-95 
school year to 903 during the 95-96 school year. Only students who were absent 10 or more 
days were included in this comparison (see Table ff 2). There were 261 students who were 
absent 10 or more days during the 94-95 school year compared to 384 students who were 
absent during the 95-96 school year. 
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Table # 2. Comparison of absenteeism before and during the school improvement plan. 



Year 


Grade 9 


Grade 10 


Grade 11 


Grade 12 


Total 


94-95 


63 


81 


58 


59 


261 


95-96 


121 


122 


73 


68 


384 



Office referrals for discipline is another category on which data was collected (see 
Table # 3). There were 970 students who were referred to the office during the 94-95 school 
year compared to 1086 students who were referred during the 95-96 school year. A major 
increase in referrals occurred for the grade 10 students, and there was also a major decrease 
in referrals for grade 11 and grade 12 students. 

Table # 3. Comparison of office referrals for discipline before and during the SIP. 



Year 


Grade 9 


Grade 10 


Grade 11 


Grade 12 


Total 


94-95 


95 


143 


459 


273 


970 


95-96 


96 


448 


372 


170 


1086 



The data on suspensions shows a marked improvement (see Table # 4). In-school 
suspensions decreased from 522 during the 94-95 school year to 259 during the 95-96 school 
year. Short term suspension decreased from 104 during the 94-95 school year to 79 during 
the 95-96 school year. Expulsions, on the other hand, increased from two to 14 during the 
95-96 school year. 

Table # 4. Comparison of suspensions before and during the SIP. 



Year 


In-school Suspension 


Short-term suspension 


Expulsion 


94-95 


522 


104 


2 


95-96 


259 


79 


14 




5 



Discussion 



We are at a loss to explain the increase in absenteeism, except to say that the school 
improvement plan did not improve absenteeism. Perhaps there would have been some 
improvement if the data had been collected on all absences instead of those students absent 
10 or more days. 

In discussing the data on referrals and suspension with school officials, it was learned 
that there was a different assistant principal during the 95-96 school year. Grade 10 students 
were a particularly unruly class that was not referred as grade 9 students because teachers 
perceived that the assistant principal then did not deal effectively with discipline. This 
probably accounted for the difference with the school improvement plan having a negligible 
effect if any at all. It was the feeling of the staff that discipline had improved during the 95- 
96 school year, because the new assistant principal dealt with student discipline problems. 
This could have accounted for more referrals since teachers believed something would 
happen. It would also have accounted for fewer suspensions because problems were dealt 
with before suspension was needed. Students who were habitual offenders were expelled and 
not returned to the classroom. 

Discipline is also a component of school climate. If discipline really had improved, it 
should show up in the school climate data and it did. The school climate variable affected is 
called "order" (see Appendix C). While the improvement is not significant, it is an 
improvement from a score of 24.1 to a score of 25.4. 

Pre- and post-data were collected on school climate to measure progress on this 
objective (see Appendix C). School climate data were collected on the following variables: 
group openness, group trust, order, leadership, instruction, environment, involvement, 
expectation, and collaboration. Pre-school improvement plan data were collected in October 
1995, and post school improvement plan data were collected in May of 1996. The results 
show improvement on all variables comprising school climate— see Appendix C. The largest 
gains were in the areas of group openness and environment. The least change was in the 
variables order and involvement. A score of 28 means that the faculty agreed that the 
behaviors which measure a variable were present. A score of 21 means that half the faculty 
disagreed that the behaviors are present. The overall climate score, which combines the 
scores for all nine variables, is a score of 26.7. compared to a pre-school improvement plan 
score of 25.1. Overall, while there is room to improve, it would appear that the objective 
for improving school climate was met. 

Further information on school climate was collected through interviews with selected 
staff. While the previous quantitative data was collected at the end of the 95-96 school year, 
the following qualitative data was collected at the start of the 96-97 school year. Since a 
great deal of the staff development occurred over the summer, there was a need to collect 
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staff reactions to further evaluate the school improvement plan. 

Ten teachers were selected who were thought to be representative of the faculty. 

They were asked 14 questions (see Appendix D) and told that their responses were strictly 
confidential. Keep in mind that the school improvement plan required each department to 
learn one new teaching strategy. 

Question one asked how they felt when they were told that they were going to be 
required to learn a new teaching technique. Five responded negatively with "they did not 
like it" and the other five responded positively with comments like "excited" and "a good 
thing— things were too static." 

The responses to questions two and three which dealt with how they felt about each 
department being given a choice about the teaching strategy they wanted to learn were mostly 
positive. Only two negative responses were noted. One was that "we never really had a 
choice" and the other was that "we did not know we had a choice. " 

Question four asked if things got better as the year went on. Three teachers said no 
and the rest felt that there was improvement, e.g., "people made a conscious effort to make 
it better". One teacher said that many teachers took offense at being told they had to change. 

Questions five and six dealt with the best and worst things that happened last year. 

The positive responses dealt with the more open communications and the seminars they 
attended. Other positive responses dealt with the kids and the teacher methodology training. 
One teacher dealt with both best and worst by saying "the realization that what we had been 
doing for years was no longer going to be tolerated." Other responses to the worst thing 
dealt with annual evaluations where administrators indicated areas needing improvement. 
Comments about the tension, the leadership, and the disappointing test scores were also 
mentioned. 

Question seven dealt with what would have made things better. Three responses dealt 
with less pressure and if the change had not been dictated. The remaining responses dealt 
with improved communication, less backstabbing, and people should be more positive. 

The responses to the question concerning the involvement of the university professor 
were mostly positive indicating that this gave them some direction. Two indicated that the 
involvement was neither a help nor a hindrance. There were no negative responses. 

Questions nine, ten, and eleven dealt with the training they received, whether they 
were ready to try it, and if they were looking forward to it. All responses but one indicated 
that the training was practical, and that they were ready to try it and were looking forward to 
the new school year. One teacher said the training was redundant, but was ready to try it, 
and was looking forward to a good school year. 
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The next question dealt with what they were hearing from colleagues. All but two 
said that they were hearing positive things. One teacher said the feedback was mixed, and 
the other teacher said there was a faction that was negative and skeptical. 

Regarding the question dealing with whether the changes would improve test scores, 
only three teachers believed this. Many teachers believe that test scores will only improve if 
the attitude of parents and students improves. Teachers perceive a great deal of apathy in the 
community toward academic achievement. 

The last question addressed staff perceptions about school based decision making. All 
teachers but three felt that they had more of a say in what went on in the building. There 
was a feeling expressed by one teacher that they were pawns of the central office and the 
other two stated "no" to this question. A part of the school climate survey also dealt with 
teacher involvement in the decision making process. While the score on collaboration 
improved from 24.1 to 25.6. the score on the behavior "teachers need to be allowed to 
participate more in the decision making process" did not change. Consequently, we would 
have to infer that the goal on this objective was not achieved. 

Conclusions 



Based on the results of this data and the analysis, we concluded that progress was 
made on the goal of improving test scores and the quality of instruction. Improvement has 
also been made in improving school climate. Based on teacher feedback secured during the 
interviews and the school climate data, it would appear that discipline has improved although 
more students are being sent to the office. The area that still needs improvement is teacher 
perceptions of their involvement in the decision-making process. The committees are in 
place and there is school based decision making, but many teachers believe they can only 
make those decisions permitted by the administration and the central office. 

The general tone among the staff as a result of the school improvement plan was very 
positive. The true impact will take place during the 96-97 school year. Teachers did not 
receive training on the new teaching strategies until the summer of 1996. Consequently, 
there was no impact of this training on the quantitative data collected at the end of the 95-96 
school year. The qualitative data collected through the interviews after the start of the 96-97 
school year was a measure of the training over the summer and a further check on school 
climate. This data was very positive and encouraging. There was a great deal of doubt at 
the start of the project that the school improvement plan would be successful because of 
staff resistance to being told they had to change. Two factors, in our opinion, reduced 
resistance: 1) allowing each department to choose a teaching methodology allowed some 
ownership and reduced psychological reactance; and 2) the combined effect of increasing 
involvement in shared decision-making and the seminars on human relations, conflict 
management, and group management. The seminars gave some staff members a better 
understanding of what was happening as they experienced the changes taking place. The 
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increase in shared decision-making gave them some feeling of control, although many still 
felt that they could only do what the administration allowed. 



Limitations of the study 

The delay from the State Department of Education in releasing the funds for this 
project severely handicapped school officials in carrying out the school improvement plan. 
The true impact of the school improvement plan will not be known until the end of the 96-97 
school year when this data will be collected again. Also, the research design would be 
greatly improved if individual students were used as the unit of analysis instead of the 
school. For example, when school data on achievement were used it appeared that 
improvement had occurred. However, when individual students were used as the unit of 
analysis, it became very clear that while improvement had occurred, it was only a small 
improvement. Using the student as the unit of analysis increases the validity of the data 
comparisons. 
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Appendix D 

Harris County Survey 
8-16-96 

My role is to try and describe what happened at Harris County last year. Will you help me 
with that? 

1. How did you feel when you were told that the grant had been funded and you were 
going to be required to learn a new teaching technique? 

2. Did the fact that you were given a choice regarding which technique you could choose 
make you feel any better? 

3. How did you feel about the whole department doing the same thing? 

4. As the year went on did you see things getting better or worse? did the climate 
improve? 

5. What was the best thing that happened last year? 

6. What was the worst thing that happened last year? 

7. What would have made the year better? 

8. Was my involvement a help or a hindrance? 

9. What did you think of the training you received? 

10. Do you think you are ready to try a new teaching technique? 

1 1 . Are you looking forward to the new school year? 

12. What do you hear from your colleagues? positive or negative? 

13. Do you think the changes that have taken place will improve test scores? 
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